{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Multi Label Classification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multi-hot labels\n",
    "\n",
    "Multi-hot label is a simple extension of the one-hot label: In one-hot label, only the maximum value is selected. While in multi-hot label, the top $ n $ labels are selected at once. It's useful when all the labels are independent of one another, in such case, multi-hot labels can be treated as many independent binary classifiers (every label is a yes/no)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deep learning based methods\n",
    "\n",
    "The best deep learning based multi-classification method is to run a `Seq2Seq` model over the input, and convert that input into a sequence of labels. It sounds stupid, I know, but sometimes it performs surprisingly well."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Why are Seq2Seq sometimes more preferable to Multi-hot labels?\n",
    "\n",
    "Because multi-hot models (treating every label as a binary classification problem) don't take into account the relationships between labels.\n",
    "\n",
    "If we treat a multi-classification problem as several different binary classification problems, in most cases it works well, but in cases where the labels are correlated, it works less well.\n",
    "\n",
    "Example question:\n",
    "\n",
    "When classifying an image, we may have several labels: [apples, oranges, banana, lemon]. We want our model to output two labels to describe the current picture at hand. Suppose we are treating each label as a different binary classification problem.\n",
    "\n",
    "Example case:\n",
    "\n",
    "We show the model a picture of a banana, the model could tell us the probabilities of each label as [8%, 23%, 98%, 67%]. In this case, the model says that the picture shows a picture that looks is a banana \"and\" lemon. But how can a fruit be both a banana and lemon? The model is not making any sense!\n",
    "\n",
    "However, if the problem is about what the fruit looks like, then banana \"or\" lemon is probably a good answer."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
